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Ik BSTIi ACT 

A total of 10 studies since 1962 pertaining directly 
to the use of cloze procedure as a teaching technique are initially 
reviewed. The reviewed literature is then synthesized to cetecmine 
what is currently known about cloze as a teaching device and to 
identify general weaknesses in this research area. General problems 
identified are no real teaching, lack of focus, weak experimental 
designs, measurement problems, and omissions in reporting studies. It 
is concluded (1) that research evidence does not indicate the cloze 
procedure to be an effective teaching technique and (2) that past 
research in this area has been rather narrowly conceived and limited 
in scope. The author also feels that little attention has b'^ 
focused on appropriate instructional strategies, methods '' . 
presentation, and relative effects of varying deletion sys jr 
benefits derived by different age or grade levels. Based on the 
problems identified, suggestions are made as to the direction future 
research migJat take, and guidelines for reporting cloze research are 
offered. This publication is one of the “Reading Information Series: 
Where Bo We Go?," designed to strengthen the research in reading 
education, A bibliography is included. (AW) 
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This nen; series of ERIC/Ck WR+IRA publication is designed to 
accomplish an amazing feat: review the past, assess the present, 
and predict the future. This first publication reflects carejid- and 
thoughtful development of the series by Dr. Richard A. Earle, 
who assumed the leadership role in initiating the series. 

James L. Laffey 
Director of ERIC/CRIER 



The International Reading Association attempts, through its publica- 
tions to provide a forum for a wide spectrum of opinion on reading. 
This policy permits divergent viewpoints without assuming the 
endorsement of the Association. 

The Cloze Procedure as a Teaching Technique was prepared pursu- 
ant to a contract with the Office of Education, U.S. Department of 
Health, Education, and Welfare. Contractors undertaking such 
projects under Government sponsorship are encouraged to express 
freely their judgment in professional and technical matters. Points of 
view or opinions do not, therefore,, necessarily represent official 
office of Education position or policy. 
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ERIC/CRIER and IRA are concerned with several types of informa- 
tion analysis and their dissemination to audiences with specific 
professional needs. Among these is the producer of research-the 
research specialist, the college professor, the doctoral student. It is 
primarily to this audience that the present series is directed, although 
others may find it useful as well. Therefore the focus will rest clearly 
on the extension of research and development activities: “Where do 
we go?” Our intent is not to provide a series of exhaustive reviews of 
literature. Nor do we intend to publish definitive statements which 
will meet with unanimous approval. Rather, we solicit and present 
the thoughtful recommendations of those researchers who' : experi- 
ence and expertise has led them to fir' , .iidcijdp' ,. , i 

on problems in reading research. 

The purpose of this series of pubhcations is to strengthen the 
research which is produced in reading education. We believe that the 
series will contribute helpful perspectives on the research literature 
and stimulating suggestions to those who perform research in reading 
and related fields. 



Richard A. Earle 
Series Editor 



Since 1953, the body of literature pertaining to the cloze procedure 
has grown substantially. During this period, rhost ot the research has 
focused on three areas: (1) cloze as a technique for measuring 
reading comprehension; (2) cloze as a quantitative measure of 
readability; and (3) cloze as a tool for investigating language 
variables. A number of writers and researchers have also recom- 
mended the cloze procedure as a suitable teaching device. Their 
recommendations are based on the assumption that by going through 
the task of completing cloz^ unV' a read 'nsi '’ts mto the 

noc contexi,, recognx;mg tiie interrelatioiiships of 

language, and consequendy improving con.prehension skills. Very 
little research has been conducted using the cloze prcc^dure as a 
teaching technique. 

The purpose of this paper is threefold. The author willr'- st ;ritically 
review the literature pertaining directly to the use cf cloze as a 
teaching technique. A number of related studies in whrch doze has 
been used as a measure of comprehension following other kstruction 
will be deliberately excluded so that full attention is focused on cloze 
as a means of instruction in itself. 

Secondly, an attempt will be made to organize and syrdiesize the 
literature that is reviewed in order to determine wha currently 
known about cloze as a teaching device and to ideru fy general 
weaknesses in this research area. 

Thirdly, based on the problems and weaknesses identCied in section 
two. suggestions will be offered as to the direction i.utnre research 
might take in this area. At this point, related studies and opinion will 
be cited when they add to our knowledge of dc'ze aiiu. lend further 
evidence to research needs. 
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Review of the Literature 

Wilson L. Taylor is generally credited with being the “father” of the 
cloze procedure. In originating the cloze procedure, Taylor drew 
upon Miller’s work in communication theory, Osgood’s “disposi- 
tional mechanisms,” and the principles of statistical random sam- 
pling. His definition of cloze, which has also been accepted by most 
others working with the cloze procedure, considers cloze “a method 
of intercepting a message from a ‘transmitter’ (writer or speaker), 
mutilating its language patterns by deleting parts, and so admin' 

"ng it to ‘receivers’ (readers and listeners) that their attempts to _ 
the patterns whole again potentially yield a considerable number of 
cloze unit^.” (1953, p. 416). 

Much of the past research has concentrated on cloze as a measure- 
ment device. Many studies have examined the validity and reliability 
of cloze as a measure of comprehension. Several others have 
investigated cloze a? a measure of readability. More recent efforts 
have explored the use of cloze as a measure of certain language 
variables related to reading. Relatively fev/ studies, however, have 
experimentally used cloze as a means of instruction. A chronological 
review of these studies follows. 

Roossinck (1962) was one of the first investigators to use cloze as a 
teaching technique. Using Donald E.P. Smith’s model of comprehen- 
sion, she developed a type of programmed learning procedure which 
consisted of a series of 200 cloze exercizes, or frames. The frames 
gradually increased in difficulty according to: (1) the components of 
Smith’s model— sur>mation, classification, and abstraction; and (2) 
grammatical complexity. Roossinck developed a scheme of gradual 
grammatical complexity, moving from simple sentences to more 
complex sentences to combinations thereof, and increasing the 
number of modifiers. In addition to using selections from social 
studies and science texts, written directions and excerpts from 
informational brochures were also employed. Deletions were made 
selectively, and vocabulary and concept load were purposely kept 
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low (4th grade level) so as not to interfere with comprehension. 
Using the principles of linear programmed iearnmg, subjects were 
immediately reinforced at the completion of each frame, and eacn 
subject proceeded at his own rate. In some cases, synonyms were 
scored as correct responses. The instructional package was adminis- 
tered to one sixth grade class, composed of 18 students, in two 
sittings, 100 frames being given in each session. 

Roossinck’s study must be considered exploratory. The small 
number of subjects used, and the lack of randomization make any 
generalizing dangerous. In addition, the reader is uncertain as to just 
what is being evaluated in this study— the ability of sixth graders to 
use a programmed learning format, the use of cloze to improve 
comorehension, or both. In fact, Roossinck admits that a large 
number of student errors were due to confusion over directions and 
the mechanics of the program. She reports that students were correct 
on 75-95 percent of the frames. While a high success ratio is desirable 
in programmed learning, this high performance is well above the 
proportion of successful closures found in other cloze studies. 
Perhaps the biggest Hmitation of this study is the lack of evidence 
that the frames actually improved comprehension or increased 
reading proficiency. Because the “instruction” was limited to two 50 
minute sessions, the (question is probably moot. 

The significant features of the Roossinck study were: (1) the 
development of a hierarchy of grammatical complexity; (2) a 
conscious attempt to control deletions whde gradually increasing 
exercise difficulty; and (3) the innovative linkage of the cloze 
procedure with programmed instruction. 

In another early attempt at the instructional use of cloze, Bloomer 
(1962) used the cloze procedure as a remedial teaching technique for 
college students. Of the three groups used in the study, one received 
cloze exercises based on every-tenth word deletions, a second 
proceeded with traditional remedial exercises, and the third group 
received no treatment at all. The cloze exercises used basal, science, 
and social studies materials of elementary grade levels. The selections 
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were gradui^^ed in difficulty, and a criterion of 96 percent correct 
was set for movement from one level to another. The subjects had 
access to scoring keys to check their own work after completion of 
an exercise, but synonyms were also counted correct by the 
instructor, although this aspect is not fully explained. Pre and 
post-testing with the survey section of the Diagnostic heading Test 
revealed that the group using cloze exercises increased significantly 
more in comprehension and total reading ability. Also, the achieved 
grade point averages for the cloze group were greater than the 
predicted grade point averages made at the beginning of the study, 
although the predictor variables are never mentioned. As a result of 
this study. Bloomer felt “the cloze procedure does have a positive 
effect on comprehension and college grades.” (p. 178) 

Bloomer’s conclusion appears to be overly enthusiastic. The selection 
of subjects (volunteers), experimental mortality, regression effects, 
and lack of adequate control make one skeptical of the results of this 
study. Furthermore, in neither remedial treatment, cloze or tradi- 
tional, was any deliberate teaching conducted- In both circum- 
stances, the instructor filled a clerical role only, by taking attendance 
and recording scores. Direct instruction was avoided, according to 
Bloomer, to reduce the instructor variable. One must also question 
the depth of the instructional program and its lasting effects, for in 
some cases, stiidents completed the instructional sequence in a brief 
twelve sessions. 

Friedman (1964) employed the cloze procedure in teaching foreign 
students at the University of Florida. She constructed 20 cloze 
exercises, using every-fifth word deletions, over materials from 
McCall-Crabbs Standard Test Lessons in Reading. Two experimental 
groups received two cloze exercises per week for ten weeks. Credit 
was given for synonyms, and multiple-choice comprehension ques- 
tions followed each cloze exercise. By contrast, a control group 
received four regular McCall'Crabbs lessons per week for ten weeks. 
Although both groups made gains in comprehension, there was no 
significant difference between the mean gains of the two groups. 
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However, the difference in the amount of instruction the two groups 
received, two versus four exercises per week, may have had an effect. 
As in the previous two studies, instruction is for such a brief duration 
that one questions the permanency oi results. 

Schneyer (1965) explored the effects of the cloze procedure upon 
the reading comprehension of sixth grade pupils. Two types of cloze 
exercises were used; one built on every-tcnth word deletions and the 
other on noun-verb deletions. Schneyer based his two deletion 
systems on the lexical-structural dichotomy introduced by Rankin 
(1957). Three exercises of each type were developed for each reader 
level of a basal series. The passages were 200 words in length, each 
containing 20 deletions. Two oasses were used in the study: the 
experimental class proceeded with the regular reading program, but, 
in addition, received one cloze exercise a day, alternating between 
the two types; the control group received the regular basal 
instructional program. The cloze exercises were scored the same day 
by the teacher, using exact replacement responses, and returned to 
the students with the correct choices indicated. Scores were obtained 
for each student on the California Test of Mental Maturity, Gates 
Reading Survey, and an informal word recognition test. After 
approximately 11 weeks, the students received a post-test on the 
Gates Reading Survey. An analysis of covariance, using the Gates 
pre-test as an antecedent variable, showed no significant difference 
between the two groups in comprehension. In a further analysis of 
the results, Schneyer points out “that students whose word 
recognition ability was at khe sixth reader level or above performed 
significantly better on the cloze exercises than did students whose 
word recognition ability was at fifth reader level or below.” (p. 177). 
The total results for each of the two types of cloze exercises were 
correlated with the intelligence and reading test results. Both cloze 
types correlated significantly with the language-I.Q. results. The 
tenth-word deletion system was much more highly related to 
intelligence than was the noun-verb deletion system (.63 vs. .42). 
This seems to substantiate Rankin’s earlier contention that the 



The Cloze Procedure as a Teaching Technique 



every-n^^ deletion system is more related to intelligence, whereas the 
selective deletion of nouns and verbs provides a measure of 
comprehension less influenced by intelligence. In explaining the 
results, Schneyer hypothesized that discussion of the reasons for 
selecting responses might be more effective than just checking for 
correctness. 



Contrasted with the previous studies, Schneyer’s investigation differs 
in: (1) increased exposure to cloze exercises; students responded 
daily rather than once or twice a week, and for a longer period of 
time; and (2) a more sophisticated Lceatment of the data, rather than 
a reliance on crude gain scores. The lack of randomization in the 
selection and assignment of subjects was overcome somewhat by the 
analysis of covariance technique, but still limits the generalizability 
of the findings. Another weakness was the disregard of teacher 
differences and their effect on the outcome. Perhaps cloze type— 
every-n^^ versus noun-verb deletions— could have been incorporated 
into the study as an additional factor and reduced the confounding 
effects of the experimental variable. 



In a somewhat different approach, Blumenfield and Miller (1966) 
first wanted to determine what good English students knew 
grammatically that enabled them to learn material more efficiently 
than poor .students. Secondly, they wanted to use their findings from 
the first part of the study to teach the poor students that 
information to determine if it had any effect on their reading ability. 
They used 36 passages, each 150 words in length, ranging in 
difficulty from first grade level through advanced readings in 
experimental psychology. An every -fifth deletion system was used, 
and the starting point rotated so as to produce five cloze forms, with 
a measure of difficulty of each word, for each passage. A word cla.ss 
score, based on the number of insertions that v/ere gfaniroatically 
satisfactory, regardless of their meaning, was used to ai?.alyze the 
results. The percentage of word, class corrt pie tions represented 
gra.mmatical difficulty. After analyzing hidividual word classes to 
defcernime differences, the author.*? foimd ho significarit differences ■ 
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between the performances of good and poor students on any word 
class. In fact, on some word classes, all students had 100 percent 
completions. It appeared that hy the time a student reached college, 
he had mastered the gramma;r of his language. 

In the instructional phase of the study, the authors compared pre 
and ost-scores on the Davis Reading Test for a group of remedial 
students who completed the 36 cloze passages. No significant 
improvement in reading ability was noted. As a result of thek 
findings, the authors concluded that it was fruitless to teach 
traditional grammatical concepts past the high school level. 

Once again, “instruction” is interpreted as the act of merely filling in 
cloze eLreises. Apparently, students received no feedback as to the 
quality of their performance as they progressed through the^ 
instructional package. The authors were vague on some aspects oi 
their experimental design. For example, they claim to use “standard 
matched group control procedures” (p. 753), without ever explaining 
on what variable(s) the groups were matched. Also, the dichotomy of 
good versus poor students employed in this study loses some of its 
relevance when applied to college students. The most significant 
contributions this study made did not come from its instructional 
aspect, but from the first phase of the study, by extending our 
knowledge of the structure of our language and its relationship to 

reading. 

Probably the most ambitious investigation of cloze as a teaching 
technique was a Cooperative Research Project conducted by 
Bloomer and others (1966; Heitzman & Bloomer, 1967). The authors 
hypothesized that the act of filling in a cloze unit was in itself 
intrinsically reinfcrcing for the subject— a type of non-overt rein- 
forcement." In the fust phase of their study, fifth, seventh, ninth, and 
eleventh graders receded a total of six cloze exercises in a three week 
period. In the second phase, termed “longitudinal,” ninth graders 
continued working vwo cloze exercises per week for a period of 
twelve weeks. Over 1000 students from 49 classrooms were involved 
in the study. Seven treatment conditions were developed. Twenty- 
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four passages of 600-700 words each, containing a complete episode, 
and consisting of 5th-6th grade reading levels (determined by the 
Yoakum formula) were employed. Treatments consisted of: (1) no 
deletions; (2) random deletions; and deletions of (3) nouns; (4) 
verbs; (5) modifiers— adjectives and adverbs; (6) function words— 
prepositions and conjunctions; and (7) noun determiners. Deletions 
were made on 10 percent of the passages. Each passage was followed 
by 12 comprehension questions, six factual and six inferential, which 
had been developed and validated^in a previous pilot study. Subjects 
were randomly assigned to one of seven groups. Classroom teachers 
administered the exercises to control for experimenter effects. In the 
longitudinal phase, order effects were controlled by randomly 
admi' .Ir-tering the exercises in blocks of six. 

Pre-tss' s consisted of the Differential Aptitude Test (DAT) and the 
California Test of Mental Maturity (CTMM). Three post-tests were 
given: (1) language and reading sections of the lowu Test of Basic 
Skills (ITBS); (2) a final cloze test, 1000 words in length with 
random deletions of each type, was given one and a half weeks after 
completion of the treatments; and (3) a recall comprehension test, 
composed of a random selection of multiple-choice items found at 
the end of each passage, was administered 10 weeks after completion 
of the treatment exercises. Although the analysis of the data was 
lengthy and complex, the basic finding was that “the use of 
non-overt reinforced cloze procedure does not increase reading 
ability either during the process or as a function of post-treatment 
testing” (Heitzman and Bloomer, 1967, p. 218). There was no 
significant improvement in comprehension test scores as subjects 
progressed through the cloze exercises. The number of correct 
closures was moderately related to the Iowa Tests of Basic Skills 
reading scores, only slightly related to immediate comprehension 
scores, and inversely related to recall comprehension post-test scores. 
The authors concluded that the ability to make closures was related 
to the reading ability of the subject, but the ability to answer 
immediate comprehension questions was dependent on the type of 
cloze procedure used. The results confirmed the differential effects 
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of vanning sv'iitactic deletions. In fact, the deletion of no„n 
determineis produced comprehension superior to that of mtaut 
passages. The remova of modifiers, on the other hand, distorted 
meaning and haimpered comprehension. The authors felt that the 
value of cloz ,: in teaching comprehension is directly related to tne 
method by vuich it is presented. Their suggestions for increasing tne 
effectiveness of cloze in teaching included: (1) more remforcemen: 
by the teacher for correct responses, including synonyms; and (2) 
providing a motivational scheme in that a subject’s movement 
through the exercises is contingent upon the quality of his responses. 

Probably the most significant aspect of this study was the evidence 
of differential effects of cloze exercises based on varying syntactic 
elements. At first, this may appear to contradict Blumenfield and 
Miller’s results, mentioned earlier, but the divergent results are due to 
important differences between the two studies. Bloomer scored 
responses in terms of exact replacement only, whereas Blumenfield 
and MiUer counted all “grammatically correct” responses as satisfac- 
tory. Secondly, Bloomer’s subjects were much younger than the 
college students used by Blumenfield and Miller. 

While the Bloomer study appears larger and more sophisticated, it 
suffers fiom some of the same drawbacks as previously reported 
studies. The brief duration of the cross-sectional phase of the 
study-a mere six exposures to the treat nent-is hardly enough to 
cause anything more than the most super'V-^Tl effects on the reader. 
Much, like previous studies, no actual teacning took place. The fact 
that the lack of teaching was explained by a pseudo-psychological 
rationale-no-overt reinforcement-did not make it less real. By the 
conclusion of the study, the authors were su^esting strongly the 
need for more reinforcement, feedback, and although not mentioned 
directly, more teaching. 

Martin(1968) investigated the differential effects of instrucdoii m 
transformational grammar, and the completion of cloze exercises, on 
reading, writing, and listening. Subjects were fieshmen at South 
, Carolina State College. One half of the 100 experimental subjects 
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studied Grammar 1 and Gra'-vnz:: 2, an introduction to transforma- 
tional grammar by Jacobs a d H rsenbaom. The other half of the 
experimental subjects zeceivec 16 doze emercises. The control group 
of 42 students took the regular fre .dman lEnglish composition course. 
Although students were perraitiea to dpn up for any class they 
desired, there was no significaiit: diffeisnce in SAT-Verbai scores 
among the three groups. 

The cloze passages were taken trom a- variety of content sources and 
selective deletions of lexical elements— nouns, verbs, adjectives, and 
adverbs— were made. During the first week, two multiple-choice 
alternatives, based on grammatical and semantic criteria, were given 
for each cloze blank. After the first week, no multiple-choice 
alternatives were used. The cloze training consisted of the comple- 
tion of an exercise followed by the teachef-led class discussion, where 
the students were encouraged to verbalize their answers. 

After nine weeks, the duration of the study, post-tests were given in 
reading achievement, writing, listening, ability to recognize linguistic 
structures, and language arts skhis.- Overall results indicated that both 
experimental groups made .significanti" greater gains on the Iowa 
Silent Reading Test than the control group. There were no significant 
differences, however, between the experimental groups. Miiurtin 
concluded that: (1) neither experimental treatment was superior to 
the other; and (2) verbalizing the reasons for closures did seem to be 
appropriate for instruction using the cloze procedure. 

Martin’s study marks the first real attempt to employ the cloze 
procedure in an actual teaching situation. Rather than relying merely 
on the completion of closures, students became involved in dis- 
cussing their responses and the reasons behind them. Not surprising- 
ly, Martin’s findings reversed the non-significant trend of previous 
studies. 

While the results are encoizra^^g, certain weaknesses must be noted. 
The influence of instructors was not controlled. In fact, at one point, 
Martin explains that “the teachsrs had been previously instructed in 
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transformational grammar and were more familiar and comfortable 
with it,” (p. 74) Certainly the ability and enthusiasm o- an in.-tructor 
for a particular method would have a significant impact on sjudent 
achievement. 

Although Martin credits student discussion of closures for much of 
the success of the cloze teaching method, verbalization of responses 
was not an experimental variable in the study find indeed, was not 
tested foi’ effectiveness. 

Guice (1969) conducted an investigation to determine whether a 
group of college students who received regular instruction in reading 
comprehension plus instruction in and practice with the cloze 
procedure, would make significant gains in comprehension when 
compared with a group of students v;-ho received the Regular 
comprehension instruction. The 16 cloze exercises used were based 
on every-n^^^ (rate of deletions never specified) deletions of concept 
words-nouns, verbs, adjectives or adverbs. Two points were scored 
for exact replacement and one point for synonyms. The followmg 
pre-tests were administered: (1) Cooperative English Tests, Reading 
Comprehension, (2) Guilford, MerrifieU, and Christensen Test of 
Creativity ‘‘Consequences,^^ (3) Guilford, Merrifield, Christensen, and 
Wilson Test of Creativity “Alternate Uses,” and (4) Otis Quicu- 
Scoring Test of Mental Abtiities. Analysis of variance, based on pre 
and post-testing on the Reading Com_prehension section of the 
Cooperative English Test, showed the experimental group did not 
improve significantly more than the control group. It appeared from 
the results that other factors were at play in Guice’s study. That is, 
both afternoon groups, regardless of treatment, did better than the 
morning groups. The correlation matrix reveals some interesting 
points. Cloze correlated quite low (.20) with gain scores, which is not 
too surprising since gain scores are less reliable than either pre or 
postmeasures. Secondly, cloze correlated highly (.79) with creativity 
(although the creativity measure used to obtain the coefficient is not 
mentioned). Cloze showed only a moderate relationship (.40) with 
intelligence, substantiating previously mentioned results regardmg 
selective-deletion systems. 
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An unusual aspect of this study was Guice’s approach to course 
content. He based the course content and the selection of passages 
on a content analysis of the comprehension test he used. This 
certainly represents a reversal of what is considered good practice in 
the field and must be interpreted as a pure and simple case of 
“teaching for the test.” Notable omissions in Guice’s report of this 
study are: (1) rate of deletions; (2) number of deletions per passage; 
(3) length of passages; (4) description of the “regular” instructional 
program; (5) a rationale for acceptable synonyms (syntactic versus 
semantic considerations); and (6) a description of how cloze was 
used and whether any real instruction took place. 

A fairly recent and innovative approach to the use of cloze as a 
teaching technique is that reported by Kingston and Weaver (1970). 
They examined the use of cloze-like tasks with culturally disadvan- 
taged first graders. Although the emphasis of the study was on the 
predictive power of cloze tests, cloze-like instruction was an essential 
part of the investigation. Essentially, the cloze procedure was 
combined with a language experience approach to beginning reading. 
Students were gradually introduced to cloze during their regular 
reading lessons by having lexical items deleted from their experience 
stories and having to suggest all words that would maJce sense for the 
deletions. Their contributions were discussed and related to the 
context. Initially, pictures were used as visual clues to the deleted 
items. Chalkboard stories were typed, using a primary typewriter, 
and exchanged among classes. Later, teachers read stories to the 
children, while the children read silently, attempting to fill the 
deletions. The following tests were given during the school year: (1) 
a series of cloze tests; (2) the Ginn Pre-Primer Achievement Test; (3) 
the Ginn Primary j A chievement Test; (4) the Lee-Clark Readiness 
Test; and (5) the reading, arithmetic, and language sections of the 
California Achievement Test Test-retest reliability estimates were 
obtained for all tests on a subset of the population. Four types of 
cloze tests were used: (1) “any-word cloze”— based on every-n^^ 
deletions with a total of 50 deletions, presented in five sessions with 
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ten items each; (2) multiple-choice, structural cloze -deleticn of 
function words with the deleted words paired with distractors of t:he 
same grammatical class and randomly ordered after five deletions; '3; 
multiple-choice, lexical cloze -every-fifth deletion of nouns, rr dr 
verbs, or adjectives using the same multiple-choice format menticr ed 
above; and (4) aural-reading cloze-based on random, every- 
deletions, but read orally by the teacher, while students read silen y , 
with the teacher pausing 30 seconds at each deletion while the 
students wrote in their responses. 

The various test scores were entered into a step-wise regression 
analysis using the California Reading Test as the criterion variable. 
Surprisingly, the multiple-choice, lexical cloze test was the best sinigle 
predictor (multiple R=.68) of first grade reading achievement, with 
the any-word cloze test a close second (multiple R-.73). Both of 
thes*? cloze tests proved to be better predictors, given the intervals of 
administration, than the standardized readiness test. Other tests 
added significantly more predictive power, but the relative increases 
in percentages were comparatively low. According to Kingston and 
Weaver, “it was demonstrated in this study that first graders could 
perform on written cloze tests soon after they began reading 
(p. 13). In addition to bridging the oral-written language gap, the 
cloze technique was highly motivating to the culturally disadvan- 
taged students in this study. 

The Kingston and Weaver study is notable because it represents a 
genuine attempt to innovatively adapt the cloze procedure as a 
teaching technique. Contrary to most previous studies, cloze was 
used in a real teaching situation. Another first was the effort to 
obtain reliability estimates for the cloze instruments on comparable 
samples of students. 

While the results of this study are encouraging, there is no direct 
evidence that the use of the cloze procedure contributed to in-creased 
reading proficiency. Indeed, this was not the purpose of thi- study. 
We may infer, however, that cloze, as used in this study, showed 
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promise as an instructional technique. The multiple-choice format 
employed in this study is reminiscent of the technique used by 
Gallant (1964), but the use of grammatical class in the selection of 
distractors is a decided advantage. One wonders, however, whether 
the multiple-choice modification departs considerably from the 
theory upon which the cloze procedure is based. How does this 
change in format alter the task of the reader? Lastly, one must be 
cautious in comparing the predictor variables used in this study 
because of the time difference in the administration of the tests. For 
example, the readiness test was given during the first week of school, 
but the cloze tests were administered in February and March. While 
it may be appropriate to say that cloze tests add considerably to the 
prediction of first grade reading achievement, it may not , be 
appropriate to claim that cloze tests are better predictors than the 
readiness test without considering the rime interval involved. 

A total of nine studies pertaining directly to the use of cloze as a 
teaching technique have been reviewed in this section. In the 
following section, these studies will be considered as a collective 
group, in an effort to synthesize our current state of knowledge and 
to pinpoint problems common to this research area. 

Problems of Past Cloze Research in Teaching 

Contrary to the recommendations frequently made by authorities in 
the field, the research evidence, at the present time, does not suggest 
that the cloze procedure is an effective teachir^ technique. For the 
most part, independent studies, across a range of age levels, have 
demonstrated that the cloze procedure, used either as a supplement 
to or in lieu of “regular” reading instruction, does not produce 
significantly improved results m reading proficiency. Table 1 
provides a capsule summary of the relevant studies. Before dismissing 
the cloze procedure as a teaching technique, some of the common 
problems that have plagued past studies will be identified, and, in the 
final section of this paper, attention will be turned to future research 
efforts that will confront these problems. 



Table 1 Summary of cloze teaching studies 





Author 


Type of population Type of deletion system 


Results 




Roossinck 

(1962) 


One sixth grade 
clacs (n=18) 


Programmed format with 
selective deletions 


No real test 
of the effectivness 
of cloze 




Bloomer 

(1962) 


Three groups of 
college students 
(n=127) 


Every-lOth deletion 
system with a 96% 
criterion for movement 
through passages 


Significant differences 
in reading comprehension 
in favor of group using 
the cloze procedure 




Friedman 

(1964) 


Three groups of 
foreign students 
at university level 


Every-5th deletion 
system on 20 passages 
from McCalUCrabbs 


No significant 
difference in reading 
comprehension 


f 


Schneyer 

(1965) 


Two sixth grade 
classes (n=66) 


(1) Every-lOth word 
deletions 

(2) Noun-verb deletions 


No significant difference in 
reading comprehension 


t 

r 

1 

i: ■' 

i 

r 

v' 


Blumenfield 
& Miller 
(1966) 

rloomer, et.aL 
(1966) 
Hcitzman 
Bloomer 
(1967) 


One class of 
freshman college 
English students 
(n=20) 

Cross-section of 

5th, 7th, 9th and 
ITth graders 
(n^lOOO plus) 
Longitudinal study 
of 9th graders 
(n=125) 


Every- 5th deletion 
system, rotated to 
produce five forms 

Non-overt reinforced, 
with 10% deletions of 
(1) random, (2) nouns, 
(3) verbs, (4) modifiers, 

(5) function words, 

(6) noun determiners, 
and (7) a control passage 
of no deletions 


No significant 
difference in 
reading ability 

No significant differences 
in reading comprehension 

Evidence of differential 
effects due to varying 
syntactic deletions 


1' 

r 


MiMTtin 

(1968) 


Three groups of 
college freshmen 
(n=142) 


Selective deletion of lexi*. 
cal elements— nouns, 
verbs, adjectives, and ad- 
verbs; Some use of 
multiple-choice format 


Significant difference in 
reading ability in favor 
of group using the cloze 
control group 
Evidence that verbalizing 
the reasons for closures is 
effective 


V 

L 

i. 

{ 


Guice 

(1969) 


Four groups of 
college students 
(n=26) 


Every-n^^ deletion of 
concept words— nouns, 
verbs, adjectives, adverbs 


No significant differences 
in reading comprehension 


Kingston & 
Weaver 
(1970) 


Culturally disad- 
vantaged first 
graders (n=74) 


(1) Any-word cloze- 
random, every-n^^, 

(2) Multiple-choice, 
structural cloze— 


Cloze-iike tasks can 
be used effectively with 
first graders 



function words 

(3) Multiplc'choice 
lexical doze— nouns, 
verbo, adjectives 

(4) Aural-reading cloze— 
random, every-n deletions 



Cloze tests can serve as 
effective predictors of 1st 
grade reading achievement 



ERIC 
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Although only a small number of studies comprise the body of 
literature directly related to cloze teaching, these studies seem to 
share some common problems. To be sure, some of their weaknesses 
are similar to those of methodology studies, in general, in reading. 
However, the intention here is to focus on particular weaknesses 
relevant to studies using cloze as a teaching technique. 

1. No real teaching. It appears that the thoughts of past investi- 
gators have bordered between over-optimism and naivete. ^For the 
most part, investigators have not incorporated any direct instruction 
with the use of cloze. Instead, these investigators have relied on the 
cloze procedure itself to do the work. They have felt that students 
could improve their reading ability simply by going through the 
process of completing cloze exercises. In those exceptions where 
instruction did take place (Martin, 1968; Kingston & Weaver, 1970), 
the results were promising. The absence of direct teaching alone 
might well account for the lack of significant differences in some of 
the studies. 

In addition, the periods of instruction, insofar as they existed, were 
extremely brief, sometimes as few as six exposures to cloze exercises. 
It is doubtful that any teaching technique, regardless of its quality, 
could produce anything more than superficial effects under such a 
brief trial. 

2. Lack of focus. One reason for the absence of direct instruction 
may be that the investigators seemed to lack a lucid definition of the 
problem. They seldom exhibited a clear notion of intent. On the 
contrary, they employed the cloze procedure in hopes of improving 
“reading comprehension,” or that monolith— “general reading abil- 
ity.” Virtually no one has considered the use of cloze with other 
aspects of reading such as vocabulary, or the use of context clues. 

3. Weak experimental designs. The lack of a clear fccus and a true 
grasp of the problem leads to experimental designs which are vague 
and inadequate and to failure to control relevant experimental 
variables. In some studies, variables were confounded. For example, 
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deletion systems, in some cases, were confounded with the method 
of presentation. In such situations, the researcher is uncertain as to 
which factor is responsible for the resulting effects. In other studies, 
relevant variables such as time spent on instruction and the effects of 
the instructor, were not incorporated into the design and, conse- 
quendy, not controlled. In several of the studies, lack of randomness 
was apparent in the selection of subjects and in their assignment to 
treatment conditions. External validity was seriously restricted in 
many studies because of the highly select samples of subjects— often 
taken from a class the investigator was teaching at the time. 

4. Measurement problems. The measurement problems involved in 
the studies reported center around the difficulties in assessing 
growth. Quite naturally, the worth of a teaching method will be 
judged in terms of the growth it can produce in student achievement. 
However, the validity and reliability of the measures used to assess 
growth must be considered carefully. Most of the studies relied on 
standardized tests which purportedly measured reading comprehen- 
sion or general reading ability. It is questionable whether most of 
these tests vahdly measured the skUls students were taught during the 
instructional programs. In order to select a valjf? measurement 
instrument, an investigator must have defined carefully the reading 
behaviors he is seeking to develop in the instructional program. The 
lack of focus, a problem mentioned earlier, made this type of 
selection difficult. 

There was a tendency to use crude gains, measured by the post minus 
pretest technique, as an assessment of growth. This is probably one 
of the most unreliable v/ays to measure student progress. Crude gains 
are particularly vulnerable to the effects of regression. When college 
subjects are selected because they are participatu^ in a reniedial 
reading program and presumably are at the extreme of the 
distribution, gains are likely to occur which are not due to the 
treatment. 

5. Omissions in reporting studies. Research studies must be reported 
completely and accurately if readers are to make sound judgments in 
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Future Directions for Researcli 
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section, 
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■ ‘ ;searcli has shown that merely 
sigb:...Mdre deliberate teaching is 
necessary, But what form should such teaching take? Some investi- 
gators have su|ested: that discussions be iised after the completion 
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selecting responses (Mneyer,l%5; 1966; Martin, 
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useful approach would: be discussions during the completion process. 
Smal groups or classes could cooperatively work through the cloze 
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Kingston and Weaver’s .study {W^i In addition to offering 
instructional value, individual and group discussions can be diagno^ 
ticaly usefolby providing insights into the mental processes students 
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0 »ito!MMe'beeiused;(RoQSsiiick, 1962; Bloomer, 1962), they 
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have not been isolated as exoeiimental vgaiables., so we know little 
about how such criteria affect the use of cloze. Geitainly the 
investigator using a performaxice criterion v/ill need an ample suppl}' 
of cloze exercises. Repetition of the same passage quickly loses its 
effectiveness (Rothkopf, 1968). 

Another fundamental issue tliat' has never; really been explorsd m 
instructional studies is the method pf presenting cloze. Four 
presentation methods are coinmoii .in the literature, but only the first 
two have been used in instructional studies: 

(1) complete cloze exercise 

(2) complete cloze exercise-^ answer multiple-choice ciuesuoris 

(3) read intact passage-y complete cloze exercise 

(4) complete pre-cloze-»read passage-^ complete post-cloze ■ 

Surely the reader’s task varies with each method, but how? Whicli 
method is best suited for iiistructionaJ purposes? Future studies 
might weH examine the differeheef/aniong these methods. 

Scoring systems are another vital aspect of die mstructional use of 
cloze. Most authorities recommend accepting synonyms.; ,m addition 
to exact replacements, as coifect lor tesclmig purposes. For 
purposes, however, acceptance c4 responses of the sarne grammatical 
class may also be appropnate., -as ivr teaching the .strirctur of the 
language. Perhaps future researchers might explore a “shifting scoring 
system.” Such a system could begin by accepting responses of the 
same grammatical class, the most lenient approach, gradually move 
to accepting synonyms, and finally require exact replacement, as 
students became more adept with the cloze procedure. 

Finally, if teaching studies using cloze aj'e to be effective, they must 
be more intense and of longer duration than they generally have been 
in the past. One or two brief exposures per week for a three to 
twelve week period seem hardly adequate. The cloze procedure must 
become an integral part of the iiuTriu^tionai program. Future studies 
must be more longitudinsl lasting at least one semester to a, year if 
they are to produce substaBiial and iasdhg efich'ts. . , ■ 
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2. More direct focus on the problem. Future researchers need to 
spell out exactly what skills they are attempting to develop. For 
research purposes, it is not enough to say one is interested in 
improving reading comprehension. Specifically, what kinds of 
comprehension will be emphasized? Many past researchers seem to 
have employed a kind of “shotgun” approach; they give cloze 
exercises and then hope for some kind of reading improvement. To 
increase their chances of observing improvement, they give tests which 
measure a variety of reading skills. Many areas have not been 

explored. 

Future researchers should know why they are using their particular 
deletion system with the cloze procedure, and explain their reasons 
in their report. The every-n^^ deletion system, which assumes that 
because of semi-random sampling, a representative number of 
grammatical elements will be deleted in each passage, has been 
widely used in measuring readability. In previous instructional 
studies, many researchers have adopted the lexical-structural 
dichotomy suggested by Rankin (1957; 1959b). This division 
assumes that passages comprising lexical deletions measure the 
understanding of substantive content, while structural deletions 
involve an understanding of interrelationships of ideas and are more 
highly influenced by inteU^ence. While there is some evidence in the 
literature of the psychological reality of this dichotomy, it is not as 
convincing as many would like to believe. Does the lexical-structurd 
dichotomy apply equally across all age or grade levels, or across ail 
types of reading materials? Are certain deletion systems better than 
others at developing particular reading skills? Louthan (1965) found 
that students comprehend better after reading passages with noun 
determiners deleted than they did after reading intact passages. The 
greatest loss in comprehension came from the deletion of nouns, 
verbs, and modifiers, the basic meaning carriers of ivritten language. 
There is also some evidence that the within-sentence order of 
elements has an effect on recall (Rothkopf, 1963). Tyuture 
researchers might weU begin by examining the relative aftects of 
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varying syntactic deletions upon the development of specified 
reading skills. 

Future investigations using cloze ?. teaching tool should expand to 
other types of reading skills. Deve.ct:aig the abdicy to use context 
clues, for example, would seem to be an appropriate area of 
instruction to which to apply tV' cloze procedure. The recent 
empirical development of class" -ntion schermes by which to 
categorize context clues has given feiure researchers a framework in 
which to operate (Ames, 1966; Moskowitz, 1968: Quealy, 1969). 
Dulin (1968) has verified different in difficulty between various 
context clues and also shown n.at grammatical classes function 
differently in combining with each contextual device. Rankin and 
Overholser (1969) have demonstrated that the cloze procedure can 
be used effectively to diagnose intermediate grade students’ 
sensitivity to context clues. It would appear that the cloze procedure 
could be employed to teach students the use of context clues. By 
using one of the classification schemes as a basis for organizing 
instruction, a series of cloze passages could be developed that would 
systematically and sequentially lead students to an understanding of 
specific context clues. 

No one has specifically explored the use of cloze in teaching reading 
in the content areas. It would appear that cloze might be a useful 
supplemental technique to the teaching of content subjects at the 
junior or senior high school level, if the lexical-structural dichotomy 
is valid, a lexical cloze exercise could provide a simply constructed 
pre-test of students’ knowledge of a subject area at the beginning of 
an instructional unit. Culhane (1970) has suggested the deletion of 
nouns and verbs in teaching an understanding of the factual material 
found in content areas such as science and social studies. Louthan 
(1965), on the other hand, believes the removal of function words, 
such as noun determiners, focuses the reader’s attention on larger 
units of meaning in the passage. The type of deletion system, as 
mentioned earlier, is open to question and empirical investigation. 
Various deletion systems should be systematically examined in 
various content areas to assess the teaching utility of cloze. 
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Some writers have begun to use cloze as a means of teaching language 
(Friedman, 1964;Torrey, 1969; Kingston & Wesiver, 1970). Evidence 
su^ests that cloze could be a useirul technique for acquainting 
speakers of non-standard English with the more formal structures of 
written standard English. In studying the effectiveness of pattern 
drill, Toirey considered the cloze procedure a kind of controlled 
association test which measures a subject’s sensitivity to form class. 
Kino-ston and Weaver have shown that under certain conditions cloze 
can be used for this purpose as early as ffet grade. As in teaching 
reading, future investigators using cloz^ to promote language learning 
will need to define specifically their teaching goals. 

Research using the cloze procedure for teaching purposes should 
expand to^ still other areas. Little attention has been given to using 
doze £6 develop word meaning skills. If the cloze procedure proves 
to be effective in. devdoping the use of context clues, then with some 
adaptions, it may also be successful in vocabulary development. For 
instance, students may be asked to produce the word meanings of 
selectively deleted elements, or to recognize necessary affixes needed 
for varying contexts. 

B-ankin (1959a) lias suggested some instructional uses of the cloze 
procedure for the reading: clinic By constructing cloze exercises from 
■ subject matter texts vh'kteu at various levels of difficulty, the cloze 
procedure could provide greater transfer between the remedial 
ihaiation and the i classroom. Future studies might explore other 
clinical uses of the cloze procedure. 

This second secdon has attempted to suggest future areas which 
cloze, research might examine. Future efforts should be clear in 
purpose, with teaching goals carefully defined. Researchers must be 
cognizant of the varying eihysts of different deletion systems. The 
application of cloze shcpdd expand to other instructional areas such 
as the use of context clues, vocabulcwy development, and reading in 
the content fields. ’ - ' 

3. 'Better expermentd desions. Nearly ail past research studies have 
employed tbe.cohylhiioTial experimental-control group comparisons, 
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one group receiving doze and the other receiving some ‘Teigular” 
method of instruction. Using this type of reset rch am- roach, 
investigators have puirsued the question, “Is cloze be - ir than 
teaching method X?” -A more fruitful line of inauiry would to ask 
how various deledor systems and methods of presentation utduence 
the instructional usefulness of the cloze procedure. Also useful, 
would be exploratory studies to identify those benefits whicii cloze 
instruction has to offer students. Perhaps future research efforts 
should move away from the experimental-control group approach to 
more use of factorial designs which permit the examination of more 
th-an one effect. 

In general, future researchers should utilize principles of good 
experimental design. Randomization procedures should be followed 
in the selection of subjects and in their assignment to treatments, 
when field constraints do not allow the preferred randomization, 
alternative procedures, such as an analysis of covariance, should be 
adopted. Schneyer (1965), for example, used his reading pre-test as 
an antecedent variable to equate his two groups in an analysis of 
covariance procedure. Future researchers would do well to employ 
blocking, stratifying, or leveling strategies (Glass and Stanley, 1970, 
p. 492). Such procedures have the advantage of reducing 
experimental error and making the design more sensitive to genuine 
effects. The confounding effects of past studies can be avoided by 
stratifying on relevant variables. For example, stratifying on deletion 
system (e.g., every-lOth, selective lexical deletions, etc.) or on 
method of presentation would provide a more refined comparison of 
different levels of these variables and would also permit the 
assessment of interaction effects between these variables and others 
incorporated into the design. In past studies, very few researchers 
have even leveled or stratified on reading achievement and so were 
unable to test interaction effects between cloze teaching and reading 
ability. Proper control of extraneous variables and maximization of 
experimental variables require a' clear understanding of the factors 
involved in a study and h well defined purpose. 
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4. IfYipToved iTieosurducYit pwcedutcs. 0 ice the reading beho.viors a 
researcher seeks to develop have beer, carefully definec. some 
measurement instrument must be selected which will validi / iiiCiisure 
those behaviors. Most past researchers huve relied on sta— darc-ized 
reading tests. Future researchers, if considering using stai:c. iruiized 
tests, should be critical in their selection. Does the test meas uts tiie 
reading skills emphasized in the instructional program? Is r.ue test 
timed? If students have been given cloze exercises in iinn^ned 
situations, it may be unfair to measure their reading progrero with a 
timed test. Are the norms applicable to the population? If not, it 
may be more advantageous for a researcher to use local norms. FaJT 
and Weintraub (1970) have discussed briefly some of these issuer, and 
suggested other factors which researchers should consider in selecting 
tfists to measure dependent variables. 

An alternative to standardized tests, would be the use of px’e and post 
cloze tests. Bloomer (1966) constructed a final cloze test with 
deletions equally balanced between the various grammatical classes. 
Content, style, and difficulty of material could be kept constant 
across pre and post forms. 

Future researchers should give careful consideration to tnc 
difficulties and procedures in measuring reading growth. The use of 
crude gains should be avoided in faver of more effective altsmatives. 
Davis (1961) has proposed several methods of estiniatmg the true 
amount of change for individuals or groups. He specificauy 
recommends two of the methods (one for individuaiS, the other for 
groups) for researchers interested in determing gains, Asiother 
approach to assessing reading growth is the residual gam. leehniquc 
suggested by Tracy and Rankin (1967). Reskiiial gain is tiie 
difference between a predicted post-test measure and an observed 
measure. This procedure has the effect of ecjuating subjects 
statistically on the basis of the pre-test. According to the authors, the 
residual gain technique is d.esirable because it doe, s not require that 
the tests be expressed in equal, interval scales, and it is free qi 
regression effects. - ' 
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In some of the past studies, the permanency of gains has been 
questioned because of the administration of a test after a brief 
instructional program. Future researchers might assess the long-term 
retention of gains by using a delayed post-testing procedure, similar 
to that used by Ray (1965). Ray compared the three and six month 
measure of gains, following a reading program for college students, 
with a pre-test measure to assess the retention of improvement. 

It is strongly recommended in future research efforts, where the 
assessment of reading growth is necessary, that investigators apply 
the procedures for measuring gains that have been suggested in this 
section of the paper. 

5. Guidelines for reporting cloze research. If readers of research are 
going to be able to draw sound conclusions, conciseness in reporting 
is essential. The traditional criterion is-can another investigator 
replicate this study by following the research report? Unfortunately, 
many of the research reports reviewed in this paper do not meet this 
test. In the light of the weaknesses that have been noted in reporting 
previous research, the following guidelines are offered in the hope of 
promoting effective sharing and accumulation of knowledge: 

1. Report the type of cloze procedure used. This should include the type 
of deletion, the rate of deletion, the total number of deletions, and the 
rationale for using this particular system. 

2. Describe the material upon which the cloze passages are based. This 
should include the type of content, the style, and an estimate of the 
readability level. 

3. Explain the scoring system used, e.g., exact replacements, synonyms, 
partial credit. 

4. Fully describe the subjects used in the study. Descriptions should be 
based on factors such as reading level, grade placement, sex, 
socio-economic status, geographical environment, as well as other 
relevant variables. 

5. If cloze results are to be correlated with other comprehension measures, 
describe those measures, state the types of comprehension questions 
used, and pre-validate them before use in the study. 
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Fiiiure Directions for Research 

6. If the cloze procedure is being cornpercd to other teaching me ihocii, 
describe such methods in terms oi uiatcrids, time spent in mstmctJon, 
and philosophy behind the program. 

7. If the cloze procedure is used as a measuit of performance, either 
before and/or after instruction, reliability esthnates should be obtained 
on the cloze tests used. 
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